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The Data Quality Campaign is a national, collaborative effort to encourage and support state 
policymakers to improve the collection, availability and use of high-quality education data 
and to implement state longitudinal data systems to improve student achievement. The 
campaign aims to provide tools and resources that will assist state development of quality 
longitudinal data systems, while also providing a national forum for reducing duplication of 
effort and promoting greater coordination and consensus among the organizations focusing 
on improving data quality, access and use. 


To these ends, four site visits were conducted in the spring of 2006 to state education 
agencies (SEAs) to gather information on their experiences in developing statewide 
longitudinal data systems: Florida, Utah, Virginia, and Wisconsin. 


This publication was produced by the Data Quality Campaign/National Center for 
Educational Accountability for submission under contract with the Bill & Melinda Gates 
Foundation. The report was written by Cherry Kugle and Nancy Smith and was reviewed and 
approved by the Wisconsin Department of Public Instruction. 


© Copyright 2006. Data Quality Campaign. All rights reserved. 
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WISCONSIN: Toughing it out 


The Wisconsin Department of Public Instruction (DPI) oversees 425 public school districts 
and 15 non-district sponsored charter schools in which 874,098 students were enrolled in 
the 2005-06 school year. Milwaukee Public Schools is the largest district in the state, 
enrolling 10% of the total students, while the majority of districts in the state are small and 
rural. 

History of Development 

Legislation 

When Wisconsin started on the path to collecting student level data, the staff involved were 
not sure they would be successful politically (a key issue was student privacy) but the 
climate changed in favor of this effort, due partly to the enactment of the 2001 No Child Left 
Behind act. The State Superintendent was a proponent who felt it was important to have 
the information collected. A key former assistant state superintendent helped contact and 
meet with state legislators and other interest groups expected to be opposed to building the 
system. These efforts and the commitment to protecting privacy as the number one priority 
helped convince those who had been vocally opposed in the past to see that a statewide 
student- level longitudinal system was needed and useful. By the time development began 
in Wisconsin, "support" was primarily in the form of "lack of opposition." There were no 
funds available at the state level for development and there was no infrastructure in place - 
the system in Wisconsin is being built from scratch, primarily by DPI staff, with substantial 
assistance from a vendor. 


Unique Identifiers 

Wisconsin began to assign unique student identifiers (Wisconsin Student Numbers, WSNs) 
in spring 2004 using the Wisconsin Student Number Locator System (WSLS). In addition to 
assigning WSNs to students entering Wisconsin Public Schools, the WSLS is used to help 
ensure that the WSNs stay with students as they move from school to school and district to 
district and for updating and correcting WSLS data as needed. WSNs are available to other 
entities, such as those serving students with disabilities, the Department of Health and 
Family Services, and the Department of Corrections as of 2006-2007. Under the I ndividual 
Student Enrollment System (ISES), districts began submitting student enrollment data for 
the 2004-05 school year, along with graduates and dropouts from the 2003-04 school year. 
The student identifier was integrated with assessment records beginning with the 2005-06 
school year. The ISES includes student demographic and outcome data needed to meet the 
school, district, and state report card requirements of the Elementary and Secondary 
Education Act (ESEA). 


Partnerships 

An external advisory committee (SSDEAC- State Superintendent's Education Data Advisory 
Committee) met quarterly; staff would bring issues for discussion to these meetings. The 
committee was composed of school district representatives from large and small districts 
(superintendents, school board members, and staff from research and technology, 
information technology, and program areas), staff from regional educational agencies 
(CESAs), and vendors. 

It was important to take into consideration the needs of the largest school district, 

Milwaukee Public Schools, as well as the small districts that had no experience with this type 
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of data collection and submission. Staff felt it was imperative to provide a variety of 
options, but have found it difficult to provide the necessary flexibility in handling the 
complexity of multiple district processes due to the cost and maintenance required of a 
variety of systems. 

An internal advisory committee met regularly to give guidance to the development process. 
Staff from assessment, special education, school finance, educational statistics, and 
information technology were included, along with a veteran agency staff member who is a 
former assessment consultant now assigned to special projects. A smaller group comprised 
of staff from information technology, educational statistics, and the special projects veteran 
had decision-making authority over much of the work. Some decisions, such as the final 
exit codes to be used for the dropout collection, and the business rules for calculating 
various rates, went to the Superintendent's Cabinet for discussion and approval. 

Other efforts that aided the process were: 

o Review by the Decision Support Architecture Consortium (DSAC) under the auspices 
of the Council of Chief State School Officers (CCSSO) to develop a roadmap for 
development. 

o A site visit from a consulting firm with expertise in designing and building systems. 

o Review of a variety of sources of data definitions such as state or federal statutes, 
NCES handbooks, and SIF standards. 

o Consultation with colleagues from other states to determine if they had grappled 
with a particular issue and how it was resolved. 

Implementation Issues 

Costs 

Estimated costs to the state: 

o Approximately $1.3 million in contract costs to a vendor to aid in developing the 
statewide student identifier system ($650,000) and student level enrollment data 
collection ($650,000), beginning in November 2003. 

o Approximately $1.3 million in DPI IT development and application staff contributions 
in developing the two systems over the same time period, 

o Additional undetermined costs of non- IT staff time (program area staff and upper 
management). spent in design and analysis efforts as members of design and 
oversight committees. 

o Additional undetermined costs of training for districts. 

o Maintenance costs are undetermined, but will be absorbed by the DPI; however, the 
source of those funds are currently not identified or secured. 

Costs by Year, for most recent years: 
o In FY 2004-05, $336,000: 

■ $226,000 in IT staff time (including development of a delegated 
authority application to interface with the state security system, 
testing, analysis, design, data quality checking, help desk support, and 
development of a data aggregation process); 

■ $110,000 for contract staff to help with design, analysis and testing; 

■ time and effort of 2 + FTEs of non-IT staff; and 

■ training costs for districts. 

o I n FY 2005-06 (through February), $444,200: 

■ $127,000 in IT staff time; 

■ contract costs of $72,000 in technical support for WSLS, 
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■ $25,200 in help desk support for WSLS, 

■ $110,000 for help desk email support, 

■ $110,000 for design, analysis, and quality assurance testing; 

■ time and effort of 2 FTEs of non- IT staff; and 

■ training costs for districts. 

o For FY2006-07 estimated IT staff cost to continue is $360,000 including 
maintenance and limited enhancements to meet mandates. This does not 
include non-IT staff costs that will equal or exceed the IT costs. 

Estimated costs to districts: 

Although costs to the districts are not available, a survey was conducted by the DPI to ask 
about the time commitment involved and whether additional staff were required for the 
initial data submission. Of the respondents, 9 districts indicated they had hired additional 
staff and 30 did not. Of those hiring staff, the range was from one half-time to one full-time 
employee, or the use of contracted services for a specified duration. Of those not hiring 
additional staff, the respondents indicated that the work was accomplished by extending 
due dates on other projects, putting in extra hours, paying overtime, and shifting duties 
among existing personnel. Almost all of the respondents indicated that a large time 
commitment was required and that tracking students takes a lot of effort at the local level. 

A district representative interviewed during the site visit stated that the data submission 
effort requires Ya of the time of one staff person over a 3 to 4 month period - time taken 
away from working on other responsibilities. 

Benefits and Uses of System 

Benefits 

o Savings have not been calculated, but DPI and district staff feel strongly that 

sufficient increases in efficiencies have occurred to at least partially offset the cost to 
collect and report more data. 

o A number of data collections previously compiled have been eliminated. These 
include: the dropout report; counts of graduates; retention and attendance; fall 
enrollment; and the pre-id roster collection used for coding assessment documents. 

o The next cycle of data submission will be more efficient, as Summary Reports under 
development during the current cycle are now in place. In addition, the number of 
duplicate records needing resolution is expected to be reduced in the next cycle. 

o District staff reported an increase in professional development activities on how to 
ask meaningful questions of the data. 

o Generating labels for assessment from the student enrollment data has reduced data 
entry and programming time and costs. Districts were positive about this change 
due to the decrease in work load at the district level and the improvement in data 
quality. 

o More efficient submission of federal reports is anticipated. 

Uses 

o Assessment and other data are reported at a school level via a public website; cell 
sizes of less than or equal to five students are masked. Because confidential data 
about students must be aggregated and re-aggregated in many ways, actual rules 
are more complicated. 

o Data are used to support the state focus on accountability, as well as for school 
improvement planning. 
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o School district staff with permission (granted by a district gatekeeper) can access 
individual level assessment data, obtain item-analyses by standard, or generate a 
variety of summary reports. 

o Information on individual student performance by objective and standard is used by 
some teachers and administrators to inform instructional strategies at the classroom 
level. 

o In addition to school-level growth, the new system will allow the calculation of 
student- level growth. 

o In addition to the required state and federal evaluations and reports, the data will be 
used to evaluate the effectiveness of 

o staff development and instructional strategies by Educational Technology, and 
o class size reduction under the Student Achievement Guarantee Education 
program. 

o I n response to requests for data for research, DPI staff either provides the data or 
creates reports, depending on the nature of the request. 

o Wisconsin statute prohibits using achievement data to evaluate teachers. 

o Legal counsel at DPI has determined that staff cannot go into student level data for 
research purposes unless the research is DPI -initiated. 

Lessons Learned 

Design 

o Scheduling, resources and capacity all have to come together. 

o Develop and put in place validation rules as soon as possible. Over 100 validation 
rules were created to look for inconsistencies and illogical data. The process includes 
comparisons to prior year data to look for aberrations and outliers. Progress and 
Summary Reports that allow districts to see their own data are also a part of the 
validation process. 

o Develop a process for adding, deleting, and modifying data elements on a scheduled 
basis. 

o Have open-source applications so districts won't have to pay licensing fees. 

o Be aware that it can take from one to three years to implement a change from 
design to report, depending on whether the element is being added or modified. 

Partnerships 

o Contracts with vendors need to be carefully written to address issues such as: 

o turnover in vendor staff, requiring re-education of new staff regarding the 
state's needs and requirements; 
o lack of human resources for efforts such as the help desk; 
o the need for more onsite visits by vendor staff or the requirement that 
developers work onsite; 
o firm dates for deliverables; and 

o clear communication of report requirements and complexity of reports. 

o Cross- program meetings within the department were very helpful. Having everyone 
at the table to communicate the issues and understand the payoffs led to stronger 
commitment to consolidating data. 

o Keep an active internal data steering committee, including program area staff, and 
an external advisory group. 

o More training of school district personnel was needed, an effort hampered by a lack 
of human resources to provide one-on-one training or workshops. However, districts 
have access to a user guide provided within the application, online instruction and 
tutorials, and a help desk. 
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Other Considerations 

o An aggressive time frame has pros and cons. A shorter timeline is painful but gets 
the system in place more quickly. A longer calendar for piloting the data collection 
would have allowed better feedback from districts, 
o Be aware of the need to provide for ongoing maintenance, changes, and 

enhancements. This includes not only software, but costs for annual district training, 
refresher training, and helpdesk training. 

Recommendations for Future Development 

o Continue to provide training throughout the state to create a culture of using data to 
make instructional decisions and increase usefulness of system to all districts, 
o Develop strategies to provide support for maintaining and enhancing the system, 
including hardware, software, district training, and the help desk, 
o Develop methodologies to connect teachers to students within the system, 
o Develop methodologies to connect students in the P-12 public education system to 
higher education data. 


7 


